Relevance & Research Question: Surveys sometimes include sensitive topics, e.g. sexual behavior or tax evasion. Respondents often hesitate to answer such sensitive items which results in high item non-response rates and a specific type of response error: a tendency to underreport socially undesirable and overreport desirable behavior. The randomized response technique (RRT) (Warner, 1965) is a well-known survey technique to reduce the problem of misreporting by protecting the privacy of the respondents. However, to obtain valid and reliable data, respondents have to understand and follow the technique´s instructions. Cheating detection models (e.g. Clark & Desharnais, 1998) try to identify the respondents which do not act according to the instructions of the design (and, hence, are cheating). Web surveys offer the opportunity to “observe” the respondents´ answering process by means of additional so-called paradata. In this study we present a new approach to detect cheaters using such client-side paradata (especially item response times).
Methods & Data: We conducted a web survey during the university´s open house (N=159) using the RRT to estimate the prevalence of deceiving in a partnership. To assess the individual item response times we implemented two comparable experimental situations; the classical RRT (including a sensitive question) and a similar RR design (without a sensitive question). Assuming that cheaters give quick answers without paying much attention to the content of the question we finally tested whether the individual item response times are significantly different in both settings.
Results: We found a small proportion of cheaters. The detected proportion of cheaters has an effect on the estimated proportion of people carrying the sensitive characteristic as a comparison with the unadjusted estimator shows.
Added Value: Previous research on cheating detection has focused only on the aggregated quantity and not on the individual “quality” of cheaters. The data quality of answers to sensitive questions is improved with such a cheating detection method based on an individual level. Here item response times (and other client-side paradata) could prospectively contribute to improve the estimation process.